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Two of the greatest problems of research on affective factors, and in particular, 
research on beliefs, is what and how we observe. The first difficulty is due to the lack 
of a clear terminology; but even once it has been clearly decided what to observe, it 
is not easy to put this into practice. This report describes from a theoretical point of 
view the results obtained using a new questionnaire appositely designed to overcome 
some critical points of beliefs ’ observation. 

INTRODUCTION 

The recent book on beliefs (Leder, Pehkonen & Toemer, 2002) shows a growing 
interest for this construct in mathematics education. The contributions underline the 
most important problems of research in this field: on the one hand the lack of an 
agreement on terminology (see Furinghetti & Pehkonen, 2002; Op’t Eynde, de Corte 
& Verschaffel, 2002), on the other the difficulty in designing efficient observational 
tools (see Leder & Forgasz, 2002). 

The problem of observing pupils’ conceptions is common in mathematics education 
research; as Balacheff claims (1990, p.262): 

It is not possible to make a direct observation of pupils’ conceptions related to a given 
mathematical concept; one can only infer them from the observation of pupils’ behaviors 
in specific tasks, which is one of the more difficult methodological problems we have to 
face. 

As concerns research about affective factors, and in particular the problems related to 
their observation, McLeod ( 1 992) underlines the need of a multiple approach which 
alternates qualitative methods (such as interviews and direct observations) to 
quantitative ones (such as standard questionnaires like Likert scales) and Schoenfeld 
(1992, p.364) claims that: 

The older measurement tools and concepts found in the affective literature are simply 
inadequate. 

The observational tools used in most research can substantially be grouped under five 
typologies (see McLeod D. & S., 2002): a) physiological measures, b) interviews, c) 
direct observation of subject, d) diaries, essays, etc. e) questionnaires. 

Questionnaires have been proposed in different typologies (see Leder, 1985) and 
certainly are the most used instruments because they are easy to construct, administer 
and score. But, during the last years, the limits of questionnaires that ask students 
their agreement with certain opinions have been clearly highlighted. 

First of all, with this method the beliefs that the researcher considers important 
(Munby, 1984; Eagly & Chaiken, 1998) are selected a priori', instead, open tests, 
such as essays or interviews, are much more effective in this respect. Moreover, 
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respondents generally answer these questionnaires in a context and with goals that are 
different to those experienced when they deal with mathematics. In these cases, the 
well-known difference (Schoenfeld, 1989) between beliefs expoused and beliefs in 
action is highlighted: sometimes they are contradictory beliefs which respond to 
different goals and appear in different contexts (see also Cobb, 1986), i.e. sets of 
beliefs grouped in separate non-interacting clusters. Besides, it is often arbitrarily 
assumed that certain beliefs elicit in all individuals the same emotion (see Di Martino 
& Zan, 2003): respondents to questionnaires are requested to express their agreement 
with a certain statement and their emotional disposition (like or dislike) toward the 
statement is inferred from the grade of agreement. But as Green underlines (1971, 
P-42): 

Whenever a person holds a certain belief, he must also take some attitude 1 towards that 
belief; and that attitude is always itself capable of formulation as a belief. It is a belief 
about belief. 

Last but not least, the most widely used questionnaires simply make a list of 
commonly held beliefs without considering the connection among the beliefs. On the 
contrary it is fundamental to consider the structure of belief systems 2 (i.e. not only the 
content of beliefs but also the way people held it) because taking into account the 
psychological strength of beliefs can help both overcome the mismatch between 
beliefs expoused and beliefs in action, and in the attempt to change beliefs (Cooney, 
1993). 

If the use of questionnaires is criticizable for the reasons discussed above, it is 
undeniable that the administration and the analysis of questionnaires require less time 
than those of interviews or direct observations and, at the same time, allow the 
collection of data having a higher statistical relevance. 

In a three-years Italian Project (involving many researchers) about the evolution of 
attitude towards mathematics ( Negative attitude towards mathematics: analysis of an 
alarming phenomenon for the culture in the new millennium), in addition to essays, 
interviews, direct observations and Likert scales, a questionnaire called Integrated 
Questionnaire on Beliefs (IQB) has been specifically designed. IQB attempts to take 
into account some of the criticisms to questionnaires while maintaining some of their 
positive features. Although the strength of the project lies in the possibility of 
analyzing jointly the results obtained with different instruments, this report focuses 
on the discussion of the results of IQB from a theoretical point of view. 

METHOD 

The analysis of two questionnaires used in a previous study (Di Martino & Zan, 
2002, 2003) suggested us the idea for a new questionnaire (IQB). In particular we 
wanted to take in account the complexity of the relationships among beliefs (belief 



1 He identifies attitude with emotional disposition. 

2 Green ( 1971) underlines three features of belief systems: quasi-logical structure (beliefs can be primary or derivative), 
psychologically centrality (some beliefs are more important to people than other), clusters’ isolation (sets of beliefs can 
be protected from any relationship with other sets of beliefs). 
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systems) and between beliefs and emotions. The conclusions of that study were that a 
single belief can be linked to different beliefs in different individuals, i.e. can belong 
to different belief systems, and that the same belief can elicit in different individuals 
different emotions. We suggested the hypothesis that the emotion elicited by the 
given belief is not always simply linked to the belief itself, but to the interaction 
among beliefs in the cluster containing it. 

We have chosen a belief (from now onwards it will be called statement A) from a list 
of 12 beliefs used in our previous studies: 

In mathematics there is always a reason for everything 
The reason for the choice of statement A lies in the fact that statement A was 
recurrent in children’s essays collected for previous research (obviously IQB’s 
schema can be re -proposed with the choice of another belief). 

Using statement A we have planned the following questionnaire: 

Choose the answer (Y/N) that you most agree with. 

Then follow ONLY the path signed by the arrow, and answer the subsequent 
questions. 





In your opinion is it true that in mathematics 






there is always a reason for everything? 





t 



Why did you answer in this way? 
Try to explain: 



t 



How 


do 


you 


feel 


about 


this 


characteristic of 


maths, i.e. 


that 


there 


is 


always 


a 


reason 


for 



everything? 

□ you like it 

□ you don’t like it 

□ you find it indifferent 
Try to explain why. 



Why did you answer in this way? 

Can you give an example (or more) 
of something in mathematics which 
has no reasons? 



How do you feel about this 
characteristic of maths, i.e. that it 
isn’t true that there is always a 
reason for everything? 

□ you like it 

□ you don’t like it 

□ you find it indifferent 
Try to explain why. 
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1QB has been distributed in 13 classes of state middle schools (8 classes) and high 
schools (5 classes) and we have collected 282 questionnaires (178 from middle 
school and 104 from high school). 

RESULTS AND DISCUSSION 

The analysis of IQB has been very useful both to observe students’ beliefs (together 
with the analysis of the results obtained with the other instruments used in the 
project) and for the ongoing construction of different profiles. 

The numerical data are summarized in the following table: 





Yes 


No 


Middle school 


170(104 L-26 DAO I) 


8 (7 D-l I) 


High school 


83 (46 L-12D- 25 I) 


21 (7 L-10D-4 I) 


Total 


253 (150 L-38D-65 I) 


29 (7 L-17 D-5 I) 



L = Like, D = Dislike, I = Indifferent 



First of all we can observe that all the possible profiles (i.e. Yes-Like, Yes-Dislike, 
Yes-indifferent, No-Like, No-Dislike, No-Indifferent) are present, even if the 
percentages are considerably different. This is a further confirmation that it is 
arbitrary to link a fixed emotional disposition to a certain belief: for more than 40% 
of the students the agreement with statement A does not correspond to a positive 
emotional disposition and for approximately 24% of the students the disagreement 
with A is associated with a positive emotional disposition. 

Another evidence is the high number of agreements to the statement A, but the 
difference in the percentages between high school and middle school students is 
significant: about 95.5% in middle school and about 80% in high school. Probably 
the greater autonomy of older students, who are less interested in the search for a 
right answer (i.e. an answer that they think the teacher would appreciate), can 
explain this difference in the percentages. A very important point for the 
interpretation of results is that in IQB respondents have to justify their agreement or 
disagreement. The analysis of the justifications shows that some answers of middle 
school students are influenced by commonplaces or by the belief that there is a right 
answer: 3 

6 th Grade: I think so because mathematics is not an opinion 4 

7 th Grade: Because mathematics is an exact science 



3 Some idiomatic expressions may be lost in the translation of the transcripts. 

4 It is an idiomatic expression in Italian meaning that mathematics is an exact science and therefore there is no room for 
autonomous ideas. 
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7 th Grade: Since mathematics is thinking and logics there is a reason for everything 

6 th Grade: I think it is the most correct answer 

8 th Grade: I answered this way because I have heard that it is tiue 

Other answers are based on personal school experiences and the focus is on 
mathematics as a subject matter instead of as a science: 

6 th Grade: I answered yes because I have always met problems with explained results. As 
in the case of the infinite number of numbers, this is possible because 
one can always add one to any number 

11 th Grade: I answered no because we have been taught many things without a clear 
explanation, they are that way and that is it 

9 th Grade: I answered no because sometimes we use methods which we have been taught 
since elementary school without an explanation of why things are so 

6 th Grade: I answered yes because the problems I have faced up until now have always 
had an answer to them 

The justifications to disagreement are particularly interesting above all for the request 
to produce a mathematical example that supports their disagreement with statement 
A. We can note students’ doubts and curiosities and personal epistemologies: 

12 th Grade: Why is oo*0 not 0? Since elementary school we have been taught that any 
quantity multiplied 0 is 0. Now we discover that it is not true, why? 

9 th Grade: I do not understand why a number times zero is equal to zero and not to the 
number itself. As a matter of fact if I have a thing and I multiply it by 
nothing (zero) I still have that thing, it does not disappear! 

6 th Grade: For example: why is 2+2 equal to 4 and not 3 or something else 

8 th Grade: Why are numbers infinite? 

11 th Grade: An example is the association between numbers and names or figures. Why 
do we associate the word ‘three’ to the concept of three and why do 
we represent this concept with the symbol 3 and not 7? 

Also in the case of agreement with statement A the analysis of the justification is 
meaningful because it can highlight belief systems linked to the agreement with 
statement A: 

6 th Grade: I answered yes because mathematics is difficult to understand so there must be 
an answer to all questions 

11 th Grade: Mathematics is the science of certainties: even situations which apparently 
are unreal have a proof 

9 th Grade: Because mathematics is based on explanations (although they are useless since 
I do not understand anything) 

Moreover, the request to justify both the agreement with statement A and, using 
Green’s terminology, the attitude about it can allow to single out some different 
interpretations of the statement: these misunderstandings are not rare and it is 
important to recognize them in order to interpret the results and to improve the 
instrument. 

In the case of statement A it seems to us that some students have expressed their 
agreement/disagreement towards another statement: In mathematics there are many 
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open questions without an answer (obviously this misunderstanding depends on the 
language in which the questionnaires are written: in the case of IQB the language is 
Italian): 

6 th Grade: I answered yes because mathematics still has many secrets 

Some students have expressed their disagreement with examples out of the 
mathematics context: 

6 th Grade: For example mathematics does not have answer to: ‘does space have an 
ending?’ 

In the case of the emotional disposition associated with the agreement/disagreement 
on statement A some students have simply described their emotional disposition 
towards mathematics: for example I like mathematics. Probably this can be due to a 
careless reading of the request, nevertheless it is important, as before, to recognize 
the possibility that someone answers to a different question. 

But the most important characteristic of IQB is the request to justify the emotional 
disposition associated with the agreement/disagreement to the statement A. As we 
discussed earlier, previous studies (see Zan & Di Martino 2003) suggested that the 
emotional disposition associated to a certain belief is not directly linked to that single 
belief but to a belief system containing it. In the design of IQB we hypothesized that 
asking respondents to justify the emotional disposition could give further information 
about their beliefs linked to the original belief (i.e. the belief that it is/is not true 
statement A) or about their personal epistemology: 

8 th Grade (YES-L): I like it because if we follow the rules we cannot fail 

12 th Grade (YES-I): It is indifferent to me because if my solutions to the exercises are 
correct I do not care if there is another explanation 

8 th Grade (YES-D): I do not like it because I like mysteries and unsolvable enigmas. In 
mathematics everything has an explanation, it is a boring world for 
boring people 

9 th Grade (NO-L): I like the fact that in mathematics not everything has an explanation 
because it makes it more intriguing. Besides this stimulates the desire 
to discover new things 

12 th Grade (NO-I): It is indifferent to me because I am not a mathematician 

9 th Grade (NO-D): I do not like it because it is difficult to learn rules by heart which we 
do not understand or which we cannot find with reasoning 

The request to justify the attitude highlights that there are deeply different 
motivations for answering indifferent, these typologies are not so evident using other 
instruments as semantic differential scales or Likert scales. 

In fact the answer indifferent can derive from: 

a) A like or a dislike not too marked (this is the case typically considered with 
semantic differential scales): 

9 th Grade: It is indifferent to me, not really, I mean I like the fact of getting lots of answer 
but I am not so enthusiastic about it 

b) An alternation between like and dislike depending on the contexts: 
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8 th Grade: It is indifferent to me because I like finding answers, but not to everything (...) 

I do not like it when the teacher asks me to explain everything I have 
said 

c) The belief that a personal opinion, whether right or wrong, cannot change 
anything: 

1 1 th Grade: It is indifferent to me because mathematics is that way, and even if statement 
A were false, it would be that way the same 

d) The difficulty in recognizing and expressing the personal attitude towards a certain 
belief: 

8 th Grade: I have never thought about it enough and out of the blue I do not know if I like 
or not 



CONCLUSIONS 

Many researchers have underlined the importance of a discussion about instruments 
to observe beliefs. Obviously the development of beliefs ’theory contributes to this 
debate: in particular the increasing relevance given to belief systems rather than to 
single belief forces researchers to construct adequate instruments. 

1QB was created to meet this need and to preserve some of questionnaires’positive 
characteristics (like the easiness of administring and analyzing them). 

The request of IQB is not only to express the agreement/disagreement with the 
statement A but also to motivate it. 

But the greatest innovation of IQB is the request of expressing the reasons of one’s 
emotional disposition toward the declared belief. 

This allows the researcher to highlight other beliefs linked to the declared belief and 
the psychological centrality of the declared belief, thus giving information about the 
belief system containing it. 

The results obtained when first using IQB are encouraging: analyzing the answers to 
IQB we found links between the agreement/disagreement with statement A and other 
beliefs. 

The next step is to experiment IQB’s schema with other statements: like those 
typically used in beliefs ’research. 

Obviously IQB can be improved and some changes have already been made. But 
above all, other instruments may be constructed. As a matter of fact we believe it is 
important to work in this direction both to improve the consistency of the instruments 
with the theory and, to interpret and compare the results obtained, in this field, up 
until now at best. 
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